1. Field of the Invention
The invention relates to a method and accompanying apparatus for identifying a system, either hardware and/or software, that holds a "reserve" for a shared physical device employed within illustratively a multi-processing environment.
2. Description of the Prior Art
Present mainframe computer installations generally employ so-called "shared device" architectures. Through these architectures, a group of independently operating systems, such as illustratively a number of individual processing units (processors), share a number of separate physical devices, such as disk drives or peripherals. Such sharing permits all of these processors to utilize, for example, a common peripheral such as a particular input/output (I/O) device, as well as affords each of these processors the ability to gain access to data stored on a common device, such as a direct access storage device (DASD). This, in turn, advantageously lowers the cost of the overall installation by increasing and spreading the utilization of a single device across a number of systems. While these systems often consist of a series of processing units, these systems can be formed of other hardware devices and/or even discrete software processes, such as individual operating systems executing on a common central processing unit (CPU).
In these "shared device" architectures, each device is typically controlled by an individual micro-processor based control unit. A hardware defined communication link (the end-to-end hardware portions of which are commonly referred to as a "path") exists, through a corresponding control unit, between each system (typically a CPU) and a shared device. In essence, a system accesses a shared device over a path. Certain older high end mainframes, such as early System 370 series of computers manufactured by the present assignee, may provide upwards of eight paths for each system. For these particular computers, on the device side of the control unit, the path can take the form of a dedicated physical connection between the control unit and a particular port on the shared device or a particular address on a bus that runs between the control unit and the device. On the system side of the control unit, the path is typically formed of appropriate software messages, e.g. channel control words, carried over a hardware linkage from the system to the control unit. The entire connection from the CPU to the shared device is commonly referred to as a "channel". Each channel is uniquely associated with one shared device, has a unique channel path identifier and has logically separate and distinct facilities for communicating with its attached shared device and the CPU to which that channel is attached. A channel may contain multiple sub-channels, each of which can be used to communicate with a shared device. In this instance, sub-channels are not shared among channels; each sub-channel is associated with only one path. For further details in this regard, the reader is referred to, e.g., page 13-1 of Principles of Operation--IBM System/370 Extended Architecture, IBM Publication Number SA22-7085-1, Second Edition, January 1987 (.COPYRGT. 1987, International Business Machines Corporation), which is hereinafter referred to as the "System/370 ESA POP" manual. Hence, commands that emanate from each of these systems, e.g. CPUs travel by way of their associated addressed channels to the shared device for execution thereat with responses, if any, to any such system from the device traveling in an opposite direction. The CPU can also logically connect or disconnect a shared device from a path by issuing an appropriate command, over the path, to the control unit associated with that device.
While these physical devices are shared among several different systems, each of these devices is nevertheless constrained to execute only one command at a time. Accordingly, several years ago, the art developed a so-called "Reserve/Release" technique to serialize a device across commands issued by a number of different systems.
In essence, any system, e.g. an operating system executing at one of several CPUs or one of several operating systems executing at one CPU, desiring access to a shared device would issue a "Reserve" command through a given channel linking that system to the device. The Reserve command, when executed by the associated control unit for that channel, would cause that unit to lock out, i.e. inhibit, any other channel (and all other paths used therein) and hence all other systems from gaining access to that device. Consequently, the device would only respond to and execute commands carried over a path that transported the reserve. This would permit the device to execute a string of commands for that particular system and provide resulting data (where appropriate) thereto without interruption or interference from any of the other systems capable of sharing that device. Accordingly, this particular system would be viewed as holding a "Reserve". Execution would continue in this fashion until this particular system issued a "Release" command over the channel, thereby relinquishing its reserve. In response to the Release command, this control unit would then permit any of the systems to once again issue a Reserve command over a corresponding path, hold the reserve and thereby gain sole access to the device, and so on.
To control a channel, a control unit operates according to several pre-defined so-called "allegiance" rules. In essence, these rules collectively require that once a device has been allocated to a program (particularly a so-called "channel" program) running on one system and thus an allegiance has been established between the device and the system, that particular device becomes unavailable to any other such program running on any of the other systems. As such, the "Reserve/Release" commands when issued by the respective systems to obtain, hold and relinquish the reserve, as needed, permit multiple channel programs, to atomically execute multiple I/O operations through a single shared device.
By way of background, allegiances can be of several types, of which "active", "working" and "reserve" allegiances are particularly relevant here. In essence, an "active" allegiance occurs whenever a system is actively communicating through a channel to a device. A "working" allegiance is said to be "extended" from a system to the device, i.e. this allegiance occurs, throughout the time during which active communication is occurring over a channel therebetween (in the case of an "active" allegiance) as well as during the time a command previously received over the channel is pending at the device. In the latter case, the device may be effectively disconnected from the system (so as to allow the system the ability to process other tasks) but will subsequently reconnect through the channel to finish executing the command. An example of the latter case would include a disk drive that is appropriately positioning one of its heads in preparation for executing a data read command and then supplying accessed data to a requesting system. Oftentimes, both "working" and "active" allegiances are commonly and collectively referred to as "implicit" allegiances. A "reserve" allegiance occurs as soon as a Reserve command is issued by a system to allocate a device to a system and thereby inhibit any other paths from gaining access to that device.
Historically speaking, "Reserve/Release" commands were first implemented such that only the system that issued a Reserve command to allocate a device could issue a Release command to terminate the particular allocation. While this particular "Reserve/Release" technique provided effective serialization of a single shared device among multiple systems, it soon became apparent in the art that a portion of a path, or even an entire channel through a control unit, could malfunction, i.e. fail, in such a manner that if a system had "reserved" a shared device over a path prior to the failure, then an allegiance existed between that system and the device for the duration of the failure. Furthermore, the failure condition itself could even establish an allegiance over a path even though immediately prior thereto no such allegiance existed involving that path. Consequently, the control unit, when faced with the resulting allegiance, totally inhibited any other system from gaining access to that device as long as the failure occurred. Inasmuch as the failure condition on the path precluded the particular system which held the "reserve" from communicating with the shared device, neither that one nor any other of the systems could gain access to the device throughout the duration of this condition. Hence, a symptom of such a failure was a total inability of any of the systems to access the shared device over a prolonged period of time.
In an effort to adequately handle such failures, a "Unconditional Reserve" (U/R) command was implemented in the art. This command, when issued by any system, carried over a path to an associated control unit and executed at that unit, caused that unit to strip all the allegiances from all other such paths involving that unit and establish a reserve allegiance over the specific path which carried the U/R command. As such, this permitted the system which issued the U/R command to gain control of the shared device.
However, since the U/R command could be issued by any system, not just the particular system that held the reserve at the time of a failure, use of this command to restore a shared memory device, particular a DASD, carried a significant risk of violating the integrity of data stored on that device. In particular, if the system that issued the U/R command were the same system that held the reserve and hence the same system that was then updating data stored on the shared device, then that system could simply invoke appropriate data recovery operations and, using both the device and channel program states that existed immediately prior to the failure condition, reinstate data processing through the shared device. Consequently, no loss of data integrity was likely to occur when this system resumed updating stored data. Similarly, if no system was updating data on the device at the time the failure occurred, then any system could issue the U/R command without any loss of data integrity. However, if the shared device was reserved to a different system than the one which issued the U/R command and was updating stored data for the former system, then any data updating operations undertaken by the latter system could well jeopardize the integrity of the stored data. In this instance, the latter system would effectively "steal" the reserve from the former system. However, since the latter system would have no knowledge of the updating operations undertaken by the former system, the latter system could not invoke proper data recovery procedures and thus could well overwrite or otherwise destroy data updates then in process by the former system. In addition, the former system would also be unaware of any updating or recovery operations undertaken by the latter system. Hence, whenever the former system returns to active service, this system could overwrite or otherwise destroy data stored by the latter system.
Consequently, to avoid any such loss of data integrity that resulted from a different system issuing a U/R command and "stealing" the reserve, this system first queried a computer operator as to whether it could issue this command. Before responding to this query, the operator was required to manually: inhibit all "sharing" processes, determine which specific system held the reserve, and then determine and subsequently invoke a proper data recovery procedure, appropriate to that specific system and the processing it was then undertaking, to safeguard against any loss of data integrity. Unfortunately, this manual task placed a significant burden on the operator and, to prevent the device from being allocating to any further applications, also required that all production workload on all the systems, that were not going to issue a U/R command, be terminated for a period of time. The ensuing loss of processing time often resulted in a significant monetary cost to the owner of the computer.
If this command were to be issued but a path that had carried the reserve, i e a "reserved path", had failed, then the U/R command would be issued over a different path to the shared device.
In modern mainframe computers which employ "dynamic pathing" such as the System 370/Extended Architecture (XA) and System 390 series of computers (particularly the System 390 model ES/9000-900 computer) manufactured by the present assignee, use of the U/R command still presented significant difficulties. Specifically, rather than reserving a single path between a system and a device, dynamic pathing (as taught in illustratively U.S. Pat. No. 4,455,605 issued Jun. 19, 1984 to R. L. Cormier et al and owned by the present assignee hereof, and described in, e.g., pages 13-1 through 13-9 of the System/370 ESA POP manual) permitted an entire group of such paths to be reserved (in effect "shared") between each system and a shared device. Hence, in these computers, a system accesses a shared device through a group of paths rather than just through a single path. Each such group carried an 11-byte path group identifier which was established during initialization of the system connected to that group. The corresponding identifier for each path in the group was stored in its associated control unit during execution of a "Set Path Group ID" (SPID) command. To issue this command, each such system first retrieves appropriate hardware data, converts this data into a proper 11-byte path group identifier and then transmits the command containing this identifier down each individual path from the system to the device, thereby forming a path group. The control unit, in turn, which receives this command establishes an internal correspondence between each such path and the path group identifier. Dynamic pathing also provided the ability, through a "Sense Path Group ID" (SNID) command to read the status of any path group given its identifier (ID). In response to a system issuing this command (i.e. the "issuing" system), an associated control unit would return a message providing the identifier of its associated path group and state of that group. The state information simply indicated whether that path group was currently reserved to the issuing system or to another system but, in the latter case, without specifying which other system. As such, the returned message did not specify which particular system then held the reserve. Fortunately, dynamic pathing advantageously reduced the need, in the event of a path failure, to issue a U/R command and move a reserve to a different path. In this regard, if a single path in a group failed, the reserve would simply remain with group and communication would then simply be carried by the next available path in the group. Furthermore, dynamic pathing allowed a shared device to disconnect from one path in a group and reconnect through any other path in the same group without first relinquishing and then re-establishing a reserve; however the need to issue a U/R command in dynamic pathing environments still existed, as described below.
Moreover, the systems in such modern mainframe computers can be configured, at least ideally, to extend sharing among a large number of path groups to an extremely large number of devices. Each of these systems typically utilizes a timer(s), operating through a so-called missing interrupt handler (MIH), to produce an interrupt (generally referred to as a "`start pending` missing interrupt" or simply an "MIH interrupt") to detect a failure, such as in any shared device or path group, which would likely necessitate the issuance of a U/R command. In this regard, a timer would be started once a command involving a shared device was invoked. The timer would either time out and generate an MIH interrupt once a default interval, typically 15 seconds, occurred or would be reset upon successful completion of the command, whichever came first. Each system unit periodically tested for the occurrence of this interrupt as a symptom of a failure condition in a path group or shared device.
Unfortunately, in these modern computers, a reserve could still be stolen as the result of a system issuing a U/R command, thereby once again jeopardizing data integrity. Although the Sense Path Group ID command could be issued in response to an MIH interrupt, doing so did not identify the specific system that held the reserve. Therefore, before permitting any system to issue the U/R command, the computer operator was once again constrained to: inhibit sharing throughout the entire computer, quiesce those systems which were not going to issue a U/R command, then manually query each such system in succession (by having its status displayed on a monitor at an operator console) to determine which particular system held the reserve. Once the reserve was located, the operator could determine proper corrective action, e.g. a data recovery procedure, appropriate to this system and subsequently invoke this action. While an underlying failure condition rarely occurred, nevertheless, when it did occur, the relatively large number of systems used in the computer necessitated that, to manually query each and every system, the entire computer had to be quiesced for a rather prolonged period of time. In large computer installations, this downtime tended to be extremely costly.
Therefore, a need existed in the art to greatly reduce the downtime of an entire computer installation that, in the event of a failure condition, would be required to locate a particular system holding a reserve, take appropriate corrective action and then return the device to active service.
Furthermore, an additional complication was seen to arise in that a series of MIH interrupts could occur in the absence of a failure condition, and greatly annoy the computer operator as well as cause that operator to falsely issue a series of U/R commands and jeopardize data integrity. In these situations, a device could simply be reserved for a prolonged period of time, such as illustratively during a very long data updating operation. Issuing the U/R command and "stealing" the reserve in this situation would likely jeopardize any data then being updated through the shared device. Additionally, the occurrence of one such interrupt would simply reset the associated timer which, if the device were reserved for, e.g., several minutes, would simply cause another such interrupt to be generated, and so on for as long as the device was reserved. Each such successive interrupt would require the operator to determine whether a corresponding U/R command should be issued. However, since no allegiances then needed to be reset, any U/R command that was otherwise issued during this time would simply be unnecessary and needlessly consume effort by the operator as well as jeopardize data integrity. Therefore, in an attempt to significantly reduce the number of falsely issued U/R commands, the MVS operating system was modified to automatically issue the SNID command for any path group and determine the state of that group regardless of whether that group was then reserved or not and regardless of which system then held the reserve. If the group was reserved, then issuance of this command indicated that: (a) no failure occurred but the reserve was held for a relatively long period of time, (b) the failure will be recovered by the system then holding the reserve, or (c) the failure occurred by a system that was then unknown. Accordingly, the U/R command would not be issued and no corrective action would be invoked through the requesting system. However, if the path group was not reserved anywhere as indicated by the sense data returned by the SNID command, a reserve could still be extended through that group even after this sense data was returned. Hence, in these instances, operator intervention was still required to quiesce those systems which were not going to issue a U/R command and then manually query each system to locate any one then holding the reserve and the system which failed.
While use of the Sense Path Group ID command greatly reduced the likelihood that the operator would falsely issue a U/R command, the operator was still burdened with the task of determining, through manually querying the affected systems, whether a U/R command should be issued.
Recently, in an attempt to once again reduce operator intervention, another command, i.e. the "Reset Allegiance" (R/A) command, was introduced. In particular, this command, when issued by a system and executed by a control unit, caused that unit to reset a working allegiance that had been established for the shared device. This command, when executed by the control unit, also returns a 32 byte result to the system. Hence, if an MIH interrupt occurred, thereby indicating a failure condition exists in a path group or shared device, then this command could be automatically used by that particular system that issued the SNID command to successfully free the path group or shared device from its working allegiance. To prevent this command from stealing a reserve, the R/A command does not function whenever a reserve allegiance has been established. In this instance, use of the Reset Allegiance command merely returns a 32 byte result which: (a) specifies that a path group or shared device is reserved, and (b) includes a bit that simply indicates whether the path group or shared device is reserved to either the system which issued the command or to another system but without specifically identifying the latter system. Consequently, if an MIH interrupt occurs while any system, other than the one which issued the R/A command, holds a reserve, the operator is still generally burdened with the onerous task of manually interrogating each system (or path) to locate the specific system then holding the reserve and thereafter effectuate proper corrective action.
Accordingly, a need still exists in the art for a technique (apparatus and an accompanying method) for use in illustratively a multi-processing environment to significantly minimize, if not totally eliminate, the need, as a result of a failure condition, for an operator to manually query each system in the environment to determine which system currently holds a reserve. Moreover, owing to the ability of the "Enterprise System Connection" (ESCON) architecture, developed by the present assignee, to extend device sharing over a larger number of systems (and thus provide enhanced connectivity) as had heretofore been permitted with the so-called Original Equipment Manufacturers Interface (OEMI), as essentially described above, this need for such a technique is becoming particularly acute. Advantageously, such a technique could then be used to automatically trigger proper corrective action with concomitantly substantial reductions occurring in installation downtime and operator intervention and hence in lost processing costs.